from sklearn_benchmarks.reporting.hp_match import HPMatchReporting

scikit-learn vs. ONNX Runtime

reporting = HPMatchReporting("onnx", config="config.yml")
reporting.make_report()

KNeighborsClassifier_brute_force

onnx (1.10.1) vs. scikit-learn (1.0.dev0)

All estimators share the following parameters: algorithm=brute.

	estimator	function	n_samples_train	n_samples	n_features	mean_duration_sklearn	std_duration_sklearn	n_iter	iteration_throughput	latency	n_jobs	n_neighbors	accuracy_score	mean_duration_onnx	std_duration_onnx	speedup	std_speedup	sklearn_profiling	onnx_profiling
0	KNeighborsClassifier_brute_force	fit	100000	100000	100	0.011	0.001	NaN	7.356	0.0	-1	1	NaN	15.690	0.053	0.001	0.001	See	See
3	KNeighborsClassifier_brute_force	fit	100000	100000	100	0.011	0.000	NaN	7.339	0.0	-1	5	NaN	0.279	0.010	0.039	0.039	See	See
6	KNeighborsClassifier_brute_force	fit	100000	100000	100	0.011	0.001	NaN	7.141	0.0	1	100	NaN	16.031	0.149	0.001	0.001	See	See
9	KNeighborsClassifier_brute_force	fit	100000	100000	100	0.012	0.000	NaN	6.683	0.0	-1	100	NaN	0.300	0.014	0.040	0.040	See	See
12	KNeighborsClassifier_brute_force	fit	100000	100000	100	0.012	0.000	NaN	6.787	0.0	1	5	NaN	3.613	0.038	0.003	0.003	See	See
15	KNeighborsClassifier_brute_force	fit	100000	100000	100	0.012	0.001	NaN	6.618	0.0	1	1	NaN	0.226	0.011	0.054	0.054	See	See
18	KNeighborsClassifier_brute_force	fit	100000	100000	2	0.004	0.000	NaN	0.361	0.0	-1	1	NaN	3.520	0.060	0.001	0.001	See	See
21	KNeighborsClassifier_brute_force	fit	100000	100000	2	0.004	0.000	NaN	0.391	0.0	-1	5	NaN	0.217	0.006	0.019	0.019	See	See

	estimator	function	n_samples_train	n_samples	n_features	mean_duration_sklearn	std_duration_sklearn	n_iter	iteration_throughput	latency	n_jobs	n_neighbors	accuracy_score	mean_duration_onnx	std_duration_onnx	speedup	std_speedup	sklearn_profiling	onnx_profiling
1	KNeighborsClassifier_brute_force	predict	100000	1000	100	1.979	0.116	NaN	0.000	0.002	-1	1	0.663	0.290	0.008	6.825	6.828	See	See
2	KNeighborsClassifier_brute_force	predict	100000	1	100	0.021	0.003	NaN	0.000	0.021	-1	1	1.000	15.534	0.196	0.001	0.001	See	See
4	KNeighborsClassifier_brute_force	predict	100000	1000	100	2.644	0.074	NaN	0.000	0.003	-1	5	0.757	15.930	0.161	0.166	0.166	See	See
5	KNeighborsClassifier_brute_force	predict	100000	1	100	0.023	0.003	NaN	0.000	0.023	-1	5	1.000	0.310	0.009	0.075	0.075	See	See
7	KNeighborsClassifier_brute_force	predict	100000	1000	100	2.127	0.063	NaN	0.000	0.002	1	100	0.882	0.309	0.010	6.892	6.895	See	See
8	KNeighborsClassifier_brute_force	predict	100000	1	100	0.023	0.001	NaN	0.000	0.023	1	100	1.000	16.100	0.163	0.001	0.001	See	See
10	KNeighborsClassifier_brute_force	predict	100000	1000	100	2.751	0.065	NaN	0.000	0.003	-1	100	0.882	15.778	0.033	0.174	0.174	See	See
11	KNeighborsClassifier_brute_force	predict	100000	1	100	0.023	0.002	NaN	0.000	0.023	-1	100	1.000	0.321	0.010	0.072	0.072	See	See
13	KNeighborsClassifier_brute_force	predict	100000	1000	100	2.108	0.037	NaN	0.000	0.002	1	5	0.757	0.224	0.007	9.413	9.418	See	See
14	KNeighborsClassifier_brute_force	predict	100000	1	100	0.022	0.001	NaN	0.000	0.022	1	5	1.000	3.506	0.095	0.006	0.006	See	See
16	KNeighborsClassifier_brute_force	predict	100000	1000	100	1.410	0.016	NaN	0.001	0.001	1	1	0.663	3.653	0.216	0.386	0.387	See	See
17	KNeighborsClassifier_brute_force	predict	100000	1	100	0.022	0.001	NaN	0.000	0.022	1	1	1.000	0.216	0.004	0.102	0.102	See	See
19	KNeighborsClassifier_brute_force	predict	100000	1000	2	1.599	0.020	NaN	0.000	0.002	-1	1	0.896	0.218	0.004	7.321	7.322	See	See
20	KNeighborsClassifier_brute_force	predict	100000	1	2	0.004	0.000	NaN	0.000	0.004	-1	1	1.000	3.485	0.102	0.001	0.001	See	See
22	KNeighborsClassifier_brute_force	predict	100000	1000	2	2.425	0.066	NaN	0.000	0.002	-1	5	0.922	3.457	0.044	0.702	0.702	See	See
23	KNeighborsClassifier_brute_force	predict	100000	1	2	0.006	0.002	NaN	0.000	0.006	-1	5	1.000	0.217	0.004	0.026	0.026	See	See

KNeighborsClassifier_kd_tree

onnx (1.10.1) vs. scikit-learn (1.0.dev0)

All estimators share the following parameters: algorithm=kd_tree.

	estimator	function	n_samples_train	n_samples	n_features	mean_duration_sklearn	std_duration_sklearn	n_iter	iteration_throughput	latency	n_jobs	n_neighbors	accuracy_score	mean_duration_onnx	std_duration_onnx	speedup	std_speedup	sklearn_profiling	onnx_profiling
0	KNeighborsClassifier_kd_tree	fit	1000000	1000000	10	2.355	0.099	NaN	0.034	0.0	-1	1	NaN	120.760	0.000	0.020	0.020	See	See
3	KNeighborsClassifier_kd_tree	fit	1000000	1000000	10	3.054	0.089	NaN	0.026	0.0	-1	5	NaN	2.649	0.306	1.153	1.161	See	See
6	KNeighborsClassifier_kd_tree	fit	1000000	1000000	10	3.374	0.120	NaN	0.024	0.0	1	100	NaN	119.997	0.000	0.028	0.028	See	See
9	KNeighborsClassifier_kd_tree	fit	1000000	1000000	10	3.043	0.061	NaN	0.026	0.0	-1	100	NaN	2.594	0.198	1.173	1.177	See	See
12	KNeighborsClassifier_kd_tree	fit	1000000	1000000	10	3.055	0.091	NaN	0.026	0.0	1	5	NaN	0.039	0.014	78.174	83.035	See	See
15	KNeighborsClassifier_kd_tree	fit	1000000	1000000	10	3.139	0.112	NaN	0.025	0.0	1	1	NaN	0.005	0.000	685.620	686.214	See	See
18	KNeighborsClassifier_kd_tree	fit	1000	1000	2	0.001	0.001	NaN	0.019	0.0	-1	1	NaN	0.054	0.001	0.016	0.016	See	See
21	KNeighborsClassifier_kd_tree	fit	1000	1000	2	0.001	0.000	NaN	0.026	0.0	-1	5	NaN	0.005	0.000	0.136	0.136	See	See

	estimator	function	n_samples_train	n_samples	n_features	mean_duration_sklearn	std_duration_sklearn	n_iter	iteration_throughput	latency	n_jobs	n_neighbors	accuracy_score	mean_duration_onnx	std_duration_onnx	speedup	std_speedup	sklearn_profiling	onnx_profiling
1	KNeighborsClassifier_kd_tree	predict	1000000	1000	10	0.805	1.139	NaN	0.000	0.001	-1	1	0.929	2.447	0.247	0.329	0.331	See	See
2	KNeighborsClassifier_kd_tree	predict	1000000	1	10	0.003	0.000	NaN	0.000	0.003	-1	1	1.000	117.517	0.000	0.000	0.000	See	See
4	KNeighborsClassifier_kd_tree	predict	1000000	1000	10	0.971	0.400	NaN	0.000	0.001	-1	5	0.946	121.668	0.000	0.008	0.008	See	See
5	KNeighborsClassifier_kd_tree	predict	1000000	1	10	0.003	0.000	NaN	0.000	0.003	-1	5	1.000	2.501	0.212	0.001	0.001	See	See
7	KNeighborsClassifier_kd_tree	predict	1000000	1000	10	5.202	0.717	NaN	0.000	0.005	1	100	0.951	2.507	0.280	2.074	2.087	See	See
8	KNeighborsClassifier_kd_tree	predict	1000000	1	10	0.003	0.001	NaN	0.000	0.003	1	100	1.000	119.841	0.000	0.000	0.000	See	See
10	KNeighborsClassifier_kd_tree	predict	1000000	1000	10	3.073	0.383	NaN	0.000	0.003	-1	100	0.951	121.799	0.000	0.025	0.025	See	See
11	KNeighborsClassifier_kd_tree	predict	1000000	1	10	0.004	0.001	NaN	0.000	0.004	-1	100	1.000	2.481	0.214	0.002	0.002	See	See
13	KNeighborsClassifier_kd_tree	predict	1000000	1000	10	1.627	0.364	NaN	0.000	0.002	1	5	0.946	0.005	0.001	296.925	298.171	See	See
14	KNeighborsClassifier_kd_tree	predict	1000000	1	10	0.001	0.000	NaN	0.000	0.001	1	5	1.000	0.037	0.003	0.040	0.040	See	See
16	KNeighborsClassifier_kd_tree	predict	1000000	1000	10	0.797	0.333	NaN	0.000	0.001	1	1	0.929	0.053	0.001	15.117	15.123	See	See
17	KNeighborsClassifier_kd_tree	predict	1000000	1	10	0.001	0.000	NaN	0.000	0.001	1	1	1.000	0.005	0.000	0.200	0.200	See	See
19	KNeighborsClassifier_kd_tree	predict	1000	1000	2	0.033	0.020	NaN	0.000	0.000	-1	1	0.891	0.005	0.000	7.080	7.085	See	See
20	KNeighborsClassifier_kd_tree	predict	1000	1	2	0.003	0.000	NaN	0.000	0.003	-1	1	1.000	0.035	0.001	0.083	0.083	See	See
22	KNeighborsClassifier_kd_tree	predict	1000	1000	2	0.026	0.002	NaN	0.001	0.000	-1	5	0.911	0.050	0.036	0.528	0.653	See	See
23	KNeighborsClassifier_kd_tree	predict	1000	1	2	0.002	0.000	NaN	0.000	0.002	-1	5	1.000	0.005	0.000	0.454	0.455	See	See

HistGradientBoostingClassifier_best

onnx (1.10.1) vs. scikit-learn (1.0.dev0)

All estimators share the following parameters: learning_rate=0.01, n_iter_no_change=10.0, max_leaf_nodes=100.0, max_bins=255.0, min_samples_leaf=100.0, max_iter=300.0.

	estimator	function	n_samples_train	n_samples	n_features	mean_duration_sklearn	std_duration_sklearn	n_iter	iteration_throughput	latency	accuracy_score	mean_duration_onnx	std_duration_onnx	speedup	std_speedup	sklearn_profiling	onnx_profiling
0	HistGradientBoostingClassifier_best	fit	100000	100000	100	102.936	0.0	300	0.001	0.001	NaN	0.464	0.024	221.972	222.277	See	See

	estimator	function	n_samples_train	n_samples	n_features	mean_duration_sklearn	std_duration_sklearn	n_iter	iteration_throughput	latency	accuracy_score	mean_duration_onnx	std_duration_onnx	speedup	std_speedup	sklearn_profiling	onnx_profiling
1	HistGradientBoostingClassifier_best	predict	100000	1000	100	0.163	0.003	300	0.005	0.0	0.824	0.388	0.013	0.42	0.42	See	See